{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tvOHZz8TX_cp"
   },
   "source": [
    "# Elasticsearch MCP Server for ChatGPT\n",
    "\n",
    "This notebook demonstrates how to deploy an MCP (Model Context Protocol) server that connects ChatGPT to Elasticsearch, enabling natural language queries over internal GitHub issues and pull requests.\n",
    "\n",
    "## What You'll Build\n",
    "An MCP server that allows ChatGPT to search and retrieve information from your Elasticsearch index using natural language queries, combining semantic and keyword search for optimal results.\n",
    "\n",
    "## Steps\n",
    "- **Install Dependencies**: Set up required Python packages (fastmcp, elasticsearch, pyngrok, pandas)\n",
    "- **Configure Environment**: Set up Elasticsearch credentials and ngrok token\n",
    "- **Initialize Elasticsearch**: Connect to your Elasticsearch cluster\n",
    "- **Create Index**: Define mappings with semantic_text field for ELSER\n",
    "- **Load Sample Data**: Import GitHub issues/PRs dataset\n",
    "- **Ingest Documents**: Bulk index documents into Elasticsearch\n",
    "- **Define MCP Tools**: Create search and fetch functions for ChatGPT\n",
    "- **Deploy Server**: Start MCP server with ngrok tunnel\n",
    "- **Connect to ChatGPT**: Get public URL for ChatGPT connector setup\n",
    "\n",
    "## Prerequisites\n",
    "- Elasticsearch cluster with ELSER model deployed\n",
    "- Ngrok account with auth token\n",
    "- Python 3.8+"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LuAQtd-BYfTZ"
   },
   "source": [
    "## Install Dependencies\n",
    "\n",
    "This cell installs all required Python packages: `fastmcp` for the MCP server framework, `elasticsearch` for connecting to Elasticsearch, `pyngrok` for creating a public tunnel, and `pandas` for data manipulation.\n",
    "\n",
    "**Alternative:** You can also install dependencies using the provided 'requirements.txt' file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yDRrpVfMNZpA",
    "outputId": "8b8a3d59-92d9-4763-9ca0-30fd46763c94"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dependencies installed\n"
     ]
    }
   ],
   "source": [
    "!pip install fastmcp elasticsearch pyngrok pandas -q\n",
    "print(\"Dependencies installed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "So0cgu4cYy4e"
   },
   "source": [
    "## Import Libraries\n",
    "\n",
    "Import all necessary Python libraries for building and running the MCP server, including FastMCP for the server framework, Elasticsearch client for database connections, and pyngrok for tunneling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "EVWlHusQpy9n",
    "outputId": "ea0eb9a3-1524-4016-e01f-1b2a510f5eb2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Libraries imported successfully\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import json\n",
    "import logging\n",
    "import threading\n",
    "import time\n",
    "import pandas as pd\n",
    "from typing import Dict, List, Any\n",
    "from getpass import getpass\n",
    "from fastmcp import FastMCP\n",
    "from elasticsearch import Elasticsearch\n",
    "from elasticsearch.helpers import bulk\n",
    "from pyngrok import ngrok\n",
    "from pyngrok.conf import PyngrokConfig\n",
    "\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "logger = logging.getLogger(__name__)\n",
    "\n",
    "print(\"Libraries imported successfully\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ycdj_n8DY8p8"
   },
   "source": [
    "## Setup Configuration\n",
    "\n",
    "Load required credentials from environment variables or prompt for manual input. You'll need:\n",
    "- **ELASTICSEARCH_URL**: Your Elasticsearch cluster endpoint\n",
    "- **ELASTICSEARCH_API_KEY**: API key with read/write access  \n",
    "- **NGROK_TOKEN**: Free token from [ngrok.com](https://dashboard.ngrok.com/)\n",
    "- **ELASTICSEARCH_INDEX**: Index name (defaults to 'github_internal')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.environ[\"ELASTICSEARCH_URL\"] = os.environ.get(\"ELASTICSEARCH_URL\") or getpass(\n",
    "    \"Enter your Elasticsearch URL: \"\n",
    ")\n",
    "os.environ[\"ELASTICSEARCH_API_KEY\"] = os.environ.get(\n",
    "    \"ELASTICSEARCH_API_KEY\"\n",
    ") or getpass(\"Enter your Elasticsearch API key: \")\n",
    "os.environ[\"NGROK_TOKEN\"] = os.environ.get(\"NGROK_TOKEN\") or getpass(\n",
    "    \"Enter your Ngrok Token: \"\n",
    ")\n",
    "os.environ[\"ELASTICSEARCH_INDEX\"] = (\n",
    "    os.environ.get(\"ELASTICSEARCH_INDEX\")\n",
    "    or getpass(\"Enter your Elasticsearch Index name (default: github_internal): \")\n",
    "    or \"github_internal\"\n",
    ")\n",
    "\n",
    "ELASTICSEARCH_URL = os.environ[\"ELASTICSEARCH_URL\"]\n",
    "ELASTICSEARCH_API_KEY = os.environ[\"ELASTICSEARCH_API_KEY\"]\n",
    "NGROK_TOKEN = os.environ[\"NGROK_TOKEN\"]\n",
    "INDEX_NAME = os.environ[\"ELASTICSEARCH_INDEX\"]\n",
    "\n",
    "print(\"Configuration loaded successfully\")\n",
    "print(f\"Index name: {INDEX_NAME}\")\n",
    "print(f\"Elasticsearch URL: {ELASTICSEARCH_URL[:30]}...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s5nVMK5EhdCL"
   },
   "source": [
    "## Initialize Elasticsearch Client\n",
    "\n",
    "Create an Elasticsearch client using your credentials and verify the connection by pinging the cluster. This ensures your credentials are valid before proceeding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Z4tv4KLWhe8X",
    "outputId": "aa70e0ef-f46e-4df6-f188-c024ef0e4aed"
   },
   "outputs": [],
   "source": [
    "es_client = Elasticsearch(ELASTICSEARCH_URL, api_key=ELASTICSEARCH_API_KEY)\n",
    "\n",
    "if es_client.ping():\n",
    "    print(\"Elasticsearch connection successful\")\n",
    "    cluster_info = es_client.info()\n",
    "    print(f\"Cluster: {cluster_info['cluster_name']}\")\n",
    "    print(f\"Version: {cluster_info['version']['number']}\")\n",
    "else:\n",
    "    print(\"ERROR: Could not connect to Elasticsearch\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rYfKl1uhZyLo"
   },
   "source": [
    "## Create Index with Mappings\n",
    "\n",
    "Create an Elasticsearch index with optimized mappings for hybrid search. The key field is `text_semantic` which uses ELSER (`.elser-2-elasticsearch`) for semantic search, while other fields enable traditional keyword search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YqRm1fRqZ2E-",
    "outputId": "a1e4b28f-8b19-4d59-ad02-2d7a4bfdefe0"
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    es_client.indices.create(\n",
    "        index=INDEX_NAME,\n",
    "        body={\n",
    "            \"mappings\": {\n",
    "                \"properties\": {\n",
    "                    \"id\": {\"type\": \"keyword\"},\n",
    "                    \"title\": {\"type\": \"text\"},\n",
    "                    \"text\": {\"type\": \"text\"},\n",
    "                    \"text_semantic\": {\n",
    "                        \"type\": \"semantic_text\",\n",
    "                        \"inference_id\": \".elser-2-elasticsearch\",\n",
    "                    },\n",
    "                    \"url\": {\"type\": \"keyword\"},\n",
    "                    \"type\": {\"type\": \"keyword\"},\n",
    "                    \"status\": {\"type\": \"keyword\"},\n",
    "                    \"priority\": {\"type\": \"keyword\"},\n",
    "                    \"assignee\": {\"type\": \"keyword\"},\n",
    "                    \"created_date\": {\"type\": \"date\", \"format\": \"iso8601\"},\n",
    "                    \"resolved_date\": {\"type\": \"date\", \"format\": \"iso8601\"},\n",
    "                    \"labels\": {\"type\": \"keyword\"},\n",
    "                    \"related_pr\": {\"type\": \"keyword\"},\n",
    "                }\n",
    "            }\n",
    "        },\n",
    "    )\n",
    "    print(f\"Index '{INDEX_NAME}' created successfully\")\n",
    "except Exception as e:\n",
    "    if \"resource_already_exists_exception\" in str(e):\n",
    "        print(f\"Index '{INDEX_NAME}' already exists\")\n",
    "    else:\n",
    "        print(f\"Error creating index: {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sFw3zTwpcYJq"
   },
   "source": [
    "## Load Sample Dataset\n",
    "\n",
    "Load the sample GitHub dataset containing 15 documents with issues, pull requests, and RFCs. The dataset includes realistic content with descriptions, comments, assignees, priorities, and relationships between issues and PRs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded 15 documents from dataset\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>text</th>\n",
       "      <th>url</th>\n",
       "      <th>type</th>\n",
       "      <th>status</th>\n",
       "      <th>priority</th>\n",
       "      <th>assignee</th>\n",
       "      <th>created_date</th>\n",
       "      <th>resolved_date</th>\n",
       "      <th>labels</th>\n",
       "      <th>related_pr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ISSUE-1712</td>\n",
       "      <td>Migrate from Elasticsearch 7.x to 8.x</td>\n",
       "      <td>Description: Current Elasticsearch cluster run...</td>\n",
       "      <td>https://internal-git.techcorp.com/issues/1712</td>\n",
       "      <td>issue</td>\n",
       "      <td>in_progress</td>\n",
       "      <td>medium</td>\n",
       "      <td>david_data</td>\n",
       "      <td>2025-09-01</td>\n",
       "      <td>None</td>\n",
       "      <td>[infrastructure, elasticsearch, migration, upg...</td>\n",
       "      <td>PR-598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>RFC-038</td>\n",
       "      <td>API Versioning Strategy and Deprecation Policy</td>\n",
       "      <td>Abstract: Establishes a formal API versioning ...</td>\n",
       "      <td>https://internal-git.techcorp.com/rfcs/038</td>\n",
       "      <td>rfc</td>\n",
       "      <td>closed</td>\n",
       "      <td>medium</td>\n",
       "      <td>sarah_dev</td>\n",
       "      <td>2025-09-03</td>\n",
       "      <td>2025-09-25</td>\n",
       "      <td>[api, architecture, design, rfc]</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ISSUE-1834</td>\n",
       "      <td>Add rate limiting per user endpoint</td>\n",
       "      <td>Description: Currently rate limiting is implem...</td>\n",
       "      <td>https://internal-git.techcorp.com/issues/1834</td>\n",
       "      <td>issue</td>\n",
       "      <td>closed</td>\n",
       "      <td>medium</td>\n",
       "      <td>john_backend</td>\n",
       "      <td>2025-09-05</td>\n",
       "      <td>2025-09-12</td>\n",
       "      <td>[feature, api, redis, rate-limiting]</td>\n",
       "      <td>PR-543</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ISSUE-1756</td>\n",
       "      <td>Implement OAuth2 support for external API inte...</td>\n",
       "      <td>Description: Product team requesting OAuth2 au...</td>\n",
       "      <td>https://internal-git.techcorp.com/issues/1756</td>\n",
       "      <td>issue</td>\n",
       "      <td>open</td>\n",
       "      <td>high</td>\n",
       "      <td>sarah_dev</td>\n",
       "      <td>2025-09-08</td>\n",
       "      <td>None</td>\n",
       "      <td>[feature, api, security, oauth]</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>PR-543</td>\n",
       "      <td>Implement per-user rate limiting with Redis</td>\n",
       "      <td>Description: Implements sliding window rate li...</td>\n",
       "      <td>https://internal-git.techcorp.com/pulls/543</td>\n",
       "      <td>pull_request</td>\n",
       "      <td>closed</td>\n",
       "      <td>medium</td>\n",
       "      <td>john_backend</td>\n",
       "      <td>2025-09-10</td>\n",
       "      <td>2025-09-12</td>\n",
       "      <td>[feature, redis, rate-limiting]</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RFC-045</td>\n",
       "      <td>Design Proposal: Microservices Migration Archi...</td>\n",
       "      <td>Abstract: This RFC proposes a phased approach ...</td>\n",
       "      <td>https://internal-git.techcorp.com/rfcs/045</td>\n",
       "      <td>rfc</td>\n",
       "      <td>open</td>\n",
       "      <td>high</td>\n",
       "      <td>tech_lead_mike</td>\n",
       "      <td>2025-09-14</td>\n",
       "      <td>None</td>\n",
       "      <td>[architecture, microservices, design, rfc]</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>ISSUE-1847</td>\n",
       "      <td>API Gateway returning 429 errors during peak h...</td>\n",
       "      <td>Description: Users are experiencing 429 rate l...</td>\n",
       "      <td>https://internal-git.techcorp.com/issues/1847</td>\n",
       "      <td>issue</td>\n",
       "      <td>closed</td>\n",
       "      <td>critical</td>\n",
       "      <td>john_backend</td>\n",
       "      <td>2025-09-15</td>\n",
       "      <td>2025-09-18</td>\n",
       "      <td>[bug, api, production, performance]</td>\n",
       "      <td>PR-567</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>PR-567</td>\n",
       "      <td>Fix connection pool exhaustion in API middleware</td>\n",
       "      <td>Description: Implements exponential backoff an...</td>\n",
       "      <td>https://internal-git.techcorp.com/pulls/567</td>\n",
       "      <td>pull_request</td>\n",
       "      <td>closed</td>\n",
       "      <td>critical</td>\n",
       "      <td>john_backend</td>\n",
       "      <td>2025-09-16</td>\n",
       "      <td>2025-09-18</td>\n",
       "      <td>[bug-fix, api, performance]</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>ISSUE-1889</td>\n",
       "      <td>SQL injection vulnerability in search endpoint</td>\n",
       "      <td>Description: Security audit identified SQL inj...</td>\n",
       "      <td>https://internal-git.techcorp.com/issues/1889</td>\n",
       "      <td>issue</td>\n",
       "      <td>closed</td>\n",
       "      <td>critical</td>\n",
       "      <td>sarah_dev</td>\n",
       "      <td>2025-09-18</td>\n",
       "      <td>2025-09-19</td>\n",
       "      <td>[security, vulnerability, bug, sql]</td>\n",
       "      <td>PR-578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>PR-578</td>\n",
       "      <td>Security hotfix: Patch SQL injection vulnerabi...</td>\n",
       "      <td>Description: CRITICAL SECURITY FIX for ISSUE-1...</td>\n",
       "      <td>https://internal-git.techcorp.com/pulls/578</td>\n",
       "      <td>pull_request</td>\n",
       "      <td>closed</td>\n",
       "      <td>critical</td>\n",
       "      <td>sarah_dev</td>\n",
       "      <td>2025-09-19</td>\n",
       "      <td>2025-09-19</td>\n",
       "      <td>[security, hotfix, sql]</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>PR-598</td>\n",
       "      <td>Elasticsearch 8.x migration - Application code...</td>\n",
       "      <td>Description: Updates application code for Elas...</td>\n",
       "      <td>https://internal-git.techcorp.com/pulls/598</td>\n",
       "      <td>pull_request</td>\n",
       "      <td>in_progress</td>\n",
       "      <td>medium</td>\n",
       "      <td>david_data</td>\n",
       "      <td>2025-09-20</td>\n",
       "      <td>None</td>\n",
       "      <td>[infrastructure, elasticsearch, migration]</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>ISSUE-1923</td>\n",
       "      <td>PostgreSQL query timeout in analytics service</td>\n",
       "      <td>Description: The analytics dashboard is timing...</td>\n",
       "      <td>https://internal-git.techcorp.com/issues/1923</td>\n",
       "      <td>issue</td>\n",
       "      <td>in_progress</td>\n",
       "      <td>high</td>\n",
       "      <td>david_data</td>\n",
       "      <td>2025-09-22</td>\n",
       "      <td>None</td>\n",
       "      <td>[bug, database, performance, postgresql]</td>\n",
       "      <td>PR-589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>PR-589</td>\n",
       "      <td>Add database index for analytics query optimiz...</td>\n",
       "      <td>Description: Resolves ISSUE-1923 by adding com...</td>\n",
       "      <td>https://internal-git.techcorp.com/pulls/589</td>\n",
       "      <td>pull_request</td>\n",
       "      <td>open</td>\n",
       "      <td>high</td>\n",
       "      <td>david_data</td>\n",
       "      <td>2025-09-23</td>\n",
       "      <td>None</td>\n",
       "      <td>[database, performance, postgresql]</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>ISSUE-1998</td>\n",
       "      <td>Memory leak in notification microservice</td>\n",
       "      <td>Description: Notification service consuming in...</td>\n",
       "      <td>https://internal-git.techcorp.com/issues/1998</td>\n",
       "      <td>issue</td>\n",
       "      <td>in_progress</td>\n",
       "      <td>critical</td>\n",
       "      <td>john_backend</td>\n",
       "      <td>2025-09-28</td>\n",
       "      <td>None</td>\n",
       "      <td>[bug, production, memory-leak, microservices]</td>\n",
       "      <td>PR-612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>PR-612</td>\n",
       "      <td>Fix memory leak in WebSocket notification service</td>\n",
       "      <td>Description: Resolves ISSUE-1998 memory leak c...</td>\n",
       "      <td>https://internal-git.techcorp.com/pulls/612</td>\n",
       "      <td>pull_request</td>\n",
       "      <td>in_progress</td>\n",
       "      <td>critical</td>\n",
       "      <td>john_backend</td>\n",
       "      <td>2025-09-29</td>\n",
       "      <td>None</td>\n",
       "      <td>[bug-fix, memory-leak, websocket]</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            id                                              title  \\\n",
       "0   ISSUE-1712              Migrate from Elasticsearch 7.x to 8.x   \n",
       "1      RFC-038     API Versioning Strategy and Deprecation Policy   \n",
       "2   ISSUE-1834                Add rate limiting per user endpoint   \n",
       "3   ISSUE-1756  Implement OAuth2 support for external API inte...   \n",
       "4       PR-543        Implement per-user rate limiting with Redis   \n",
       "5      RFC-045  Design Proposal: Microservices Migration Archi...   \n",
       "6   ISSUE-1847  API Gateway returning 429 errors during peak h...   \n",
       "7       PR-567   Fix connection pool exhaustion in API middleware   \n",
       "8   ISSUE-1889     SQL injection vulnerability in search endpoint   \n",
       "9       PR-578  Security hotfix: Patch SQL injection vulnerabi...   \n",
       "10      PR-598  Elasticsearch 8.x migration - Application code...   \n",
       "11  ISSUE-1923      PostgreSQL query timeout in analytics service   \n",
       "12      PR-589  Add database index for analytics query optimiz...   \n",
       "13  ISSUE-1998           Memory leak in notification microservice   \n",
       "14      PR-612  Fix memory leak in WebSocket notification service   \n",
       "\n",
       "                                                 text  \\\n",
       "0   Description: Current Elasticsearch cluster run...   \n",
       "1   Abstract: Establishes a formal API versioning ...   \n",
       "2   Description: Currently rate limiting is implem...   \n",
       "3   Description: Product team requesting OAuth2 au...   \n",
       "4   Description: Implements sliding window rate li...   \n",
       "5   Abstract: This RFC proposes a phased approach ...   \n",
       "6   Description: Users are experiencing 429 rate l...   \n",
       "7   Description: Implements exponential backoff an...   \n",
       "8   Description: Security audit identified SQL inj...   \n",
       "9   Description: CRITICAL SECURITY FIX for ISSUE-1...   \n",
       "10  Description: Updates application code for Elas...   \n",
       "11  Description: The analytics dashboard is timing...   \n",
       "12  Description: Resolves ISSUE-1923 by adding com...   \n",
       "13  Description: Notification service consuming in...   \n",
       "14  Description: Resolves ISSUE-1998 memory leak c...   \n",
       "\n",
       "                                              url          type       status  \\\n",
       "0   https://internal-git.techcorp.com/issues/1712         issue  in_progress   \n",
       "1      https://internal-git.techcorp.com/rfcs/038           rfc       closed   \n",
       "2   https://internal-git.techcorp.com/issues/1834         issue       closed   \n",
       "3   https://internal-git.techcorp.com/issues/1756         issue         open   \n",
       "4     https://internal-git.techcorp.com/pulls/543  pull_request       closed   \n",
       "5      https://internal-git.techcorp.com/rfcs/045           rfc         open   \n",
       "6   https://internal-git.techcorp.com/issues/1847         issue       closed   \n",
       "7     https://internal-git.techcorp.com/pulls/567  pull_request       closed   \n",
       "8   https://internal-git.techcorp.com/issues/1889         issue       closed   \n",
       "9     https://internal-git.techcorp.com/pulls/578  pull_request       closed   \n",
       "10    https://internal-git.techcorp.com/pulls/598  pull_request  in_progress   \n",
       "11  https://internal-git.techcorp.com/issues/1923         issue  in_progress   \n",
       "12    https://internal-git.techcorp.com/pulls/589  pull_request         open   \n",
       "13  https://internal-git.techcorp.com/issues/1998         issue  in_progress   \n",
       "14    https://internal-git.techcorp.com/pulls/612  pull_request  in_progress   \n",
       "\n",
       "    priority        assignee created_date resolved_date  \\\n",
       "0     medium      david_data   2025-09-01          None   \n",
       "1     medium       sarah_dev   2025-09-03    2025-09-25   \n",
       "2     medium    john_backend   2025-09-05    2025-09-12   \n",
       "3       high       sarah_dev   2025-09-08          None   \n",
       "4     medium    john_backend   2025-09-10    2025-09-12   \n",
       "5       high  tech_lead_mike   2025-09-14          None   \n",
       "6   critical    john_backend   2025-09-15    2025-09-18   \n",
       "7   critical    john_backend   2025-09-16    2025-09-18   \n",
       "8   critical       sarah_dev   2025-09-18    2025-09-19   \n",
       "9   critical       sarah_dev   2025-09-19    2025-09-19   \n",
       "10    medium      david_data   2025-09-20          None   \n",
       "11      high      david_data   2025-09-22          None   \n",
       "12      high      david_data   2025-09-23          None   \n",
       "13  critical    john_backend   2025-09-28          None   \n",
       "14  critical    john_backend   2025-09-29          None   \n",
       "\n",
       "                                               labels related_pr  \n",
       "0   [infrastructure, elasticsearch, migration, upg...     PR-598  \n",
       "1                    [api, architecture, design, rfc]       None  \n",
       "2                [feature, api, redis, rate-limiting]     PR-543  \n",
       "3                     [feature, api, security, oauth]       None  \n",
       "4                     [feature, redis, rate-limiting]       None  \n",
       "5          [architecture, microservices, design, rfc]       None  \n",
       "6                 [bug, api, production, performance]     PR-567  \n",
       "7                         [bug-fix, api, performance]       None  \n",
       "8                 [security, vulnerability, bug, sql]     PR-578  \n",
       "9                             [security, hotfix, sql]       None  \n",
       "10         [infrastructure, elasticsearch, migration]       None  \n",
       "11           [bug, database, performance, postgresql]     PR-589  \n",
       "12                [database, performance, postgresql]       None  \n",
       "13      [bug, production, memory-leak, microservices]     PR-612  \n",
       "14                  [bug-fix, memory-leak, websocket]       None  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file_path = \"github_internal_dataset.json\"\n",
    "df = pd.read_json(file_path)\n",
    "\n",
    "documents = df.to_dict(\"records\")\n",
    "print(f\"Loaded {len(documents)} documents from dataset\")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "80BHi4HgdSxh"
   },
   "source": [
    "## Ingest Documents to Elasticsearch\n",
    "\n",
    "Bulk index all documents into Elasticsearch. The code copies the `text` field to `text_semantic` for ELSER processing, then waits 15 seconds for semantic embeddings to be generated before verifying the document count."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "u1JgCgwldWhN",
    "outputId": "1cae2312-5508-4757-a3c8-c2a2870ba129"
   },
   "outputs": [],
   "source": [
    "def generate_actions():\n",
    "    for doc in documents:\n",
    "        doc[\"text_semantic\"] = doc[\"text\"]\n",
    "        yield {\"_index\": INDEX_NAME, \"_source\": doc}\n",
    "\n",
    "\n",
    "try:\n",
    "    success, errors = bulk(es_client, generate_actions())\n",
    "    print(f\"Successfully indexed {success} documents\")\n",
    "\n",
    "    if errors:\n",
    "        print(f\"Errors during indexing: {errors}\")\n",
    "\n",
    "    print(\"Waiting 15 seconds for ELSER to process documents...\")\n",
    "    time.sleep(15)\n",
    "\n",
    "    count = es_client.count(index=INDEX_NAME)[\"count\"]\n",
    "    print(f\"Total documents in index: {count}\")\n",
    "\n",
    "except Exception as e:\n",
    "    print(f\"Error during bulk indexing: {str(e)}\")\n",
    "    print(\"If you see timeout errors, wait a few seconds and try again\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OTg5ePAbdbCW"
   },
   "source": [
    "## Define MCP Server\n",
    "\n",
    "Define the MCP server with two tools that ChatGPT will use:\n",
    "1. **search(query)**: Hybrid search combining semantic (ELSER) and keyword (BM25) search using RRF (Reciprocal Rank Fusion). Returns top 10 results with id, title, and url.\n",
    "2. **fetch(id)**: Retrieves complete document details by ID, returning all fields including full text content and metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5aLcJzLxdeLS",
    "outputId": "3c29e137-1c9f-43ee-bb33-c18945c34680"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MCP server defined successfully\n"
     ]
    }
   ],
   "source": [
    "server_instructions = \"\"\"\n",
    "This MCP server provides access to TechCorp's internal GitHub issues and pull requests.\n",
    "Use search to find relevant issues/PRs, then fetch to get complete details.\n",
    "\"\"\"\n",
    "\n",
    "\n",
    "def create_server():\n",
    "    mcp = FastMCP(\n",
    "        name=\"Elasticsearch GitHub Issues MCP\", instructions=server_instructions\n",
    "    )\n",
    "\n",
    "    @mcp.tool()\n",
    "    async def search(query: str) -> Dict[str, List[Dict[str, Any]]]:\n",
    "        \"\"\"\n",
    "        Search for internal issues and PRs using hybrid search.\n",
    "        Returns list with id, title, and url.\n",
    "        \"\"\"\n",
    "        if not query or not query.strip():\n",
    "            return {\"results\": []}\n",
    "\n",
    "        logger.info(f\"Searching for: '{query}'\")\n",
    "\n",
    "        try:\n",
    "            # Hybrid search using RRF: combines semantic (ELSER) + keyword (multi_match) results\n",
    "            response = es_client.search(\n",
    "                index=INDEX_NAME,\n",
    "                size=10,\n",
    "                source=[\"id\", \"title\", \"url\", \"type\", \"priority\"],\n",
    "                retriever={\n",
    "                    \"rrf\": {\n",
    "                        \"retrievers\": [\n",
    "                            {\n",
    "                                # Semantic retriever using ELSER embeddings\n",
    "                                \"standard\": {\n",
    "                                    \"query\": {\n",
    "                                        \"semantic\": {\n",
    "                                            \"field\": \"text_semantic\",\n",
    "                                            \"query\": query,\n",
    "                                        }\n",
    "                                    }\n",
    "                                }\n",
    "                            },\n",
    "                            {\n",
    "                                # Keyword retriever with fuzzy matching\n",
    "                                \"standard\": {\n",
    "                                    \"query\": {\n",
    "                                        \"multi_match\": {\n",
    "                                            \"query\": query,\n",
    "                                            \"fields\": [\n",
    "                                                \"title^3\",\n",
    "                                                \"text^2\",\n",
    "                                                \"assignee^2\",\n",
    "                                                \"type\",\n",
    "                                                \"labels\",\n",
    "                                                \"priority\",\n",
    "                                            ],\n",
    "                                            \"type\": \"best_fields\",\n",
    "                                            \"fuzziness\": \"AUTO\",\n",
    "                                        }\n",
    "                                    }\n",
    "                                }\n",
    "                            },\n",
    "                        ],\n",
    "                        \"rank_window_size\": 50,\n",
    "                        \"rank_constant\": 60,\n",
    "                    }\n",
    "                },\n",
    "            )\n",
    "\n",
    "            # Extract and format search results\n",
    "            results = []\n",
    "            if response and \"hits\" in response:\n",
    "                for hit in response[\"hits\"][\"hits\"]:\n",
    "                    source = hit[\"_source\"]\n",
    "                    results.append(\n",
    "                        {\n",
    "                            \"id\": source.get(\"id\", hit[\"_id\"]),\n",
    "                            \"title\": source.get(\"title\", \"Unknown\"),\n",
    "                            \"url\": source.get(\"url\", \"\"),\n",
    "                        }\n",
    "                    )\n",
    "\n",
    "            logger.info(f\"Found {len(results)} results\")\n",
    "            return {\"results\": results}\n",
    "\n",
    "        except Exception as e:\n",
    "            logger.error(f\"Search error: {e}\")\n",
    "            raise ValueError(f\"Search failed: {str(e)}\")\n",
    "\n",
    "    @mcp.tool()\n",
    "    async def fetch(id: str) -> Dict[str, Any]:\n",
    "        \"\"\"\n",
    "        Retrieve complete issue/PR details by ID.\n",
    "        Returns id, title, text, url, and metadata.\n",
    "        \"\"\"\n",
    "        if not id:\n",
    "            raise ValueError(\"ID is required\")\n",
    "\n",
    "        logger.info(f\"Fetching: {id}\")\n",
    "\n",
    "        try:\n",
    "            # Query by ID to get full document\n",
    "            response = es_client.search(\n",
    "                index=INDEX_NAME, body={\"query\": {\"term\": {\"id\": id}}, \"size\": 1}\n",
    "            )\n",
    "\n",
    "            if not response or not response[\"hits\"][\"hits\"]:\n",
    "                raise ValueError(f\"Document with id '{id}' not found\")\n",
    "\n",
    "            hit = response[\"hits\"][\"hits\"][0]\n",
    "            source = hit[\"_source\"]\n",
    "\n",
    "            # Return all document fields\n",
    "            result = {\n",
    "                \"id\": source.get(\"id\", id),\n",
    "                \"title\": source.get(\"title\", \"Unknown\"),\n",
    "                \"text\": source.get(\"text\", \"\"),\n",
    "                \"url\": source.get(\"url\", \"\"),\n",
    "                \"type\": source.get(\"type\", \"\"),\n",
    "                \"status\": source.get(\"status\", \"\"),\n",
    "                \"priority\": source.get(\"priority\", \"\"),\n",
    "                \"assignee\": source.get(\"assignee\", \"\"),\n",
    "                \"created_date\": source.get(\"created_date\", \"\"),\n",
    "                \"resolved_date\": source.get(\"resolved_date\", \"\"),\n",
    "                \"labels\": source.get(\"labels\", \"\"),\n",
    "                \"related_pr\": source.get(\"related_pr\", \"\"),\n",
    "            }\n",
    "\n",
    "            logger.info(f\"Fetched: {result['title']}\")\n",
    "            return result\n",
    "\n",
    "        except Exception as e:\n",
    "            logger.error(f\"Fetch error: {e}\")\n",
    "            raise ValueError(f\"Failed to fetch '{id}': {str(e)}\")\n",
    "\n",
    "    return mcp\n",
    "\n",
    "\n",
    "print(\"MCP server defined successfully\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0aTU4xeedpHc"
   },
   "source": [
    "## Start Ngrok Tunnel\n",
    "\n",
    "Create a public HTTPS tunnel using ngrok to expose your local MCP server on port 8000. This allows ChatGPT to connect to your server from anywhere. Copy the displayed URL (ending in `/sse`) to use in ChatGPT's connector settings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SN-wlTOGdtIs",
    "outputId": "c78552d6-914d-4327-943f-6430d4f12569"
   },
   "outputs": [],
   "source": [
    "ngrok.set_auth_token(NGROK_TOKEN)\n",
    "\n",
    "pyngrok_config = PyngrokConfig(region=\"us\")\n",
    "public_url = ngrok.connect(8000, \"http\", pyngrok_config=pyngrok_config, bind_tls=True)\n",
    "\n",
    "print(\"=\" * 70)\n",
    "print(\"MCP SERVER IS READY!\")\n",
    "print(\"=\" * 70)\n",
    "print(f\"\\nPublic URL (use in ChatGPT): {public_url}/sse\")\n",
    "print(\"\\nIMPORTANT: Copy the URL above (including /sse at the end)\")\n",
    "print(\"\\nTo connect in ChatGPT:\")\n",
    "print(\"1. Go to Settings > Connectors\")\n",
    "print(\"2. Click 'Create' or 'Add Custom Connector'\")\n",
    "print(\"3. Paste the URL above\")\n",
    "print(\"4. Save and start using!\")\n",
    "print(\"\\nKeep this notebook running while using the connector\")\n",
    "print(\"=\" * 70)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SFfSlDVAdxe1"
   },
   "source": [
    "## Run MCP Server\n",
    "\n",
    "Start the MCP server in a background thread using SSE (Server-Sent Events) transport. The server runs on `0.0.0.0:8000` and stays alive to handle requests from ChatGPT via the ngrok tunnel. Keep this cell running while using the connector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "okQYfcJfdzp-",
    "outputId": "d226ff48-41fc-4b5b-b63d-f651a20d23d5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting MCP server...\n",
      "Server is running. To stop: Runtime > Interrupt execution\n",
      "\n",
      "Server started successfully!\n",
      "Your ngrok URL is ready to use in ChatGPT\n",
      "Keep this cell running...\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">\n",
       "\n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">╭──────────────────────────────────────────────────────────────────────────────╮</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                                                                              <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                        <span style=\"color: #00c6ff; text-decoration-color: #00c6ff\"> </span><span style=\"color: #00c3ff; text-decoration-color: #00c3ff\">▄</span><span style=\"color: #00c0ff; text-decoration-color: #00c0ff\">▀</span><span style=\"color: #00bdff; text-decoration-color: #00bdff\">▀</span><span style=\"color: #00baff; text-decoration-color: #00baff\"> </span><span style=\"color: #00b8ff; text-decoration-color: #00b8ff\">▄</span><span style=\"color: #00b5ff; text-decoration-color: #00b5ff\">▀</span><span style=\"color: #00b2ff; text-decoration-color: #00b2ff\">█</span><span style=\"color: #00afff; text-decoration-color: #00afff\"> </span><span style=\"color: #00acff; text-decoration-color: #00acff\">█</span><span style=\"color: #00a9ff; text-decoration-color: #00a9ff\">▀</span><span style=\"color: #00a6ff; text-decoration-color: #00a6ff\">▀</span><span style=\"color: #00a3ff; text-decoration-color: #00a3ff\"> </span><span style=\"color: #00a0ff; text-decoration-color: #00a0ff\">▀</span><span style=\"color: #009dff; text-decoration-color: #009dff\">█</span><span style=\"color: #009bff; text-decoration-color: #009bff\">▀</span><span style=\"color: #0098ff; text-decoration-color: #0098ff\"> </span><span style=\"color: #0095ff; text-decoration-color: #0095ff\">█</span><span style=\"color: #0092ff; text-decoration-color: #0092ff\">▀</span><span style=\"color: #008fff; text-decoration-color: #008fff\">▄</span><span style=\"color: #008cff; text-decoration-color: #008cff\">▀</span><span style=\"color: #0089ff; text-decoration-color: #0089ff\">█</span><span style=\"color: #0086ff; text-decoration-color: #0086ff\"> </span><span style=\"color: #0083ff; text-decoration-color: #0083ff\">█</span><span style=\"color: #0080ff; text-decoration-color: #0080ff\">▀</span><span style=\"color: #007eff; text-decoration-color: #007eff\">▀</span><span style=\"color: #007bff; text-decoration-color: #007bff\"> </span><span style=\"color: #0078ff; text-decoration-color: #0078ff\">█</span><span style=\"color: #0075ff; text-decoration-color: #0075ff\">▀</span><span style=\"color: #0072ff; text-decoration-color: #0072ff\">█</span>                        <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                        <span style=\"color: #00c6ff; text-decoration-color: #00c6ff\"> </span><span style=\"color: #00c3ff; text-decoration-color: #00c3ff\">█</span><span style=\"color: #00c0ff; text-decoration-color: #00c0ff\">▀</span><span style=\"color: #00bdff; text-decoration-color: #00bdff\"> </span><span style=\"color: #00baff; text-decoration-color: #00baff\"> </span><span style=\"color: #00b8ff; text-decoration-color: #00b8ff\">█</span><span style=\"color: #00b5ff; text-decoration-color: #00b5ff\">▀</span><span style=\"color: #00b2ff; text-decoration-color: #00b2ff\">█</span><span style=\"color: #00afff; text-decoration-color: #00afff\"> </span><span style=\"color: #00acff; text-decoration-color: #00acff\">▄</span><span style=\"color: #00a9ff; text-decoration-color: #00a9ff\">▄</span><span style=\"color: #00a6ff; text-decoration-color: #00a6ff\">█</span><span style=\"color: #00a3ff; text-decoration-color: #00a3ff\"> </span><span style=\"color: #00a0ff; text-decoration-color: #00a0ff\"> </span><span style=\"color: #009dff; text-decoration-color: #009dff\">█</span><span style=\"color: #009bff; text-decoration-color: #009bff\"> </span><span style=\"color: #0098ff; text-decoration-color: #0098ff\"> </span><span style=\"color: #0095ff; text-decoration-color: #0095ff\">█</span><span style=\"color: #0092ff; text-decoration-color: #0092ff\"> </span><span style=\"color: #008fff; text-decoration-color: #008fff\">▀</span><span style=\"color: #008cff; text-decoration-color: #008cff\"> </span><span style=\"color: #0089ff; text-decoration-color: #0089ff\">█</span><span style=\"color: #0086ff; text-decoration-color: #0086ff\"> </span><span style=\"color: #0083ff; text-decoration-color: #0083ff\">█</span><span style=\"color: #0080ff; text-decoration-color: #0080ff\">▄</span><span style=\"color: #007eff; text-decoration-color: #007eff\">▄</span><span style=\"color: #007bff; text-decoration-color: #007bff\"> </span><span style=\"color: #0078ff; text-decoration-color: #0078ff\">█</span><span style=\"color: #0075ff; text-decoration-color: #0075ff\">▀</span><span style=\"color: #0072ff; text-decoration-color: #0072ff\">▀</span>                        <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                                                                              <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                               <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">FastMCP 2.13.0.2</span>                               <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                                                                              <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                                                                              <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>               <span style=\"font-weight: bold\">🖥  </span><span style=\"color: #008080; text-decoration-color: #008080\">Server name: </span><span style=\"color: #7f7fbf; text-decoration-color: #7f7fbf; font-weight: bold\">Elasticsearch GitHub Issues MCP</span>                <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>               <span style=\"font-weight: bold\">   </span><span style=\"color: #008080; text-decoration-color: #008080\">             </span><span style=\"color: #7f7fbf; text-decoration-color: #7f7fbf; font-weight: bold\">                               </span>                <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>               <span style=\"font-weight: bold\">📦 </span><span style=\"color: #008080; text-decoration-color: #008080\">Transport:   </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">SSE                            </span>                <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>               <span style=\"font-weight: bold\">🔗 </span><span style=\"color: #008080; text-decoration-color: #008080\">Server URL:  </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">http://0.0.0.0:8000/sse        </span>                <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>               <span style=\"font-weight: bold\">   </span><span style=\"color: #008080; text-decoration-color: #008080\">             </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">                               </span>                <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>               <span style=\"font-weight: bold\">📚 </span><span style=\"color: #008080; text-decoration-color: #008080\">Docs:        </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">https://gofastmcp.com          </span>                <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>               <span style=\"font-weight: bold\">🚀 </span><span style=\"color: #008080; text-decoration-color: #008080\">Hosting:     </span><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">https://fastmcp.cloud          </span>                <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                                                                              <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">│</span>                  \n",
       "                 <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">╰──────────────────────────────────────────────────────────────────────────────╯</span>                  \n",
       "\n",
       "\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\n",
       "\n",
       "                 \u001b[2m╭──────────────────────────────────────────────────────────────────────────────╮\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m                                                                              \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m                        \u001b[38;2;0;198;255m \u001b[0m\u001b[38;2;0;195;255m▄\u001b[0m\u001b[38;2;0;192;255m▀\u001b[0m\u001b[38;2;0;189;255m▀\u001b[0m\u001b[38;2;0;186;255m \u001b[0m\u001b[38;2;0;184;255m▄\u001b[0m\u001b[38;2;0;181;255m▀\u001b[0m\u001b[38;2;0;178;255m█\u001b[0m\u001b[38;2;0;175;255m \u001b[0m\u001b[38;2;0;172;255m█\u001b[0m\u001b[38;2;0;169;255m▀\u001b[0m\u001b[38;2;0;166;255m▀\u001b[0m\u001b[38;2;0;163;255m \u001b[0m\u001b[38;2;0;160;255m▀\u001b[0m\u001b[38;2;0;157;255m█\u001b[0m\u001b[38;2;0;155;255m▀\u001b[0m\u001b[38;2;0;152;255m \u001b[0m\u001b[38;2;0;149;255m█\u001b[0m\u001b[38;2;0;146;255m▀\u001b[0m\u001b[38;2;0;143;255m▄\u001b[0m\u001b[38;2;0;140;255m▀\u001b[0m\u001b[38;2;0;137;255m█\u001b[0m\u001b[38;2;0;134;255m \u001b[0m\u001b[38;2;0;131;255m█\u001b[0m\u001b[38;2;0;128;255m▀\u001b[0m\u001b[38;2;0;126;255m▀\u001b[0m\u001b[38;2;0;123;255m \u001b[0m\u001b[38;2;0;120;255m█\u001b[0m\u001b[38;2;0;117;255m▀\u001b[0m\u001b[38;2;0;114;255m█\u001b[0m                        \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m                        \u001b[38;2;0;198;255m \u001b[0m\u001b[38;2;0;195;255m█\u001b[0m\u001b[38;2;0;192;255m▀\u001b[0m\u001b[38;2;0;189;255m \u001b[0m\u001b[38;2;0;186;255m \u001b[0m\u001b[38;2;0;184;255m█\u001b[0m\u001b[38;2;0;181;255m▀\u001b[0m\u001b[38;2;0;178;255m█\u001b[0m\u001b[38;2;0;175;255m \u001b[0m\u001b[38;2;0;172;255m▄\u001b[0m\u001b[38;2;0;169;255m▄\u001b[0m\u001b[38;2;0;166;255m█\u001b[0m\u001b[38;2;0;163;255m \u001b[0m\u001b[38;2;0;160;255m \u001b[0m\u001b[38;2;0;157;255m█\u001b[0m\u001b[38;2;0;155;255m \u001b[0m\u001b[38;2;0;152;255m \u001b[0m\u001b[38;2;0;149;255m█\u001b[0m\u001b[38;2;0;146;255m \u001b[0m\u001b[38;2;0;143;255m▀\u001b[0m\u001b[38;2;0;140;255m \u001b[0m\u001b[38;2;0;137;255m█\u001b[0m\u001b[38;2;0;134;255m \u001b[0m\u001b[38;2;0;131;255m█\u001b[0m\u001b[38;2;0;128;255m▄\u001b[0m\u001b[38;2;0;126;255m▄\u001b[0m\u001b[38;2;0;123;255m \u001b[0m\u001b[38;2;0;120;255m█\u001b[0m\u001b[38;2;0;117;255m▀\u001b[0m\u001b[38;2;0;114;255m▀\u001b[0m                        \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m                                                                              \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m                               \u001b[1;34mFastMCP 2.13.0.2\u001b[0m                               \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m                                                                              \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m                                                                              \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m               \u001b[1m🖥 \u001b[0m\u001b[1m \u001b[0m\u001b[36mServer name:\u001b[0m\u001b[36m \u001b[0m\u001b[1;2;34mElasticsearch GitHub Issues MCP\u001b[0m                \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m               \u001b[1m   \u001b[0m\u001b[36m             \u001b[0m\u001b[1;2;34m                               \u001b[0m                \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m               \u001b[1m📦\u001b[0m\u001b[1m \u001b[0m\u001b[36mTransport:  \u001b[0m\u001b[36m \u001b[0m\u001b[2mSSE                            \u001b[0m                \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m               \u001b[1m🔗\u001b[0m\u001b[1m \u001b[0m\u001b[36mServer URL: \u001b[0m\u001b[36m \u001b[0m\u001b[2mhttp://0.0.0.0:8000/sse        \u001b[0m                \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m               \u001b[1m  \u001b[0m\u001b[1m \u001b[0m\u001b[36m            \u001b[0m\u001b[36m \u001b[0m\u001b[2m                               \u001b[0m                \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m               \u001b[1m📚\u001b[0m\u001b[1m \u001b[0m\u001b[36mDocs:       \u001b[0m\u001b[36m \u001b[0m\u001b[2mhttps://gofastmcp.com          \u001b[0m                \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m               \u001b[1m🚀\u001b[0m\u001b[1m \u001b[0m\u001b[36mHosting:    \u001b[0m\u001b[36m \u001b[0m\u001b[2mhttps://fastmcp.cloud          \u001b[0m                \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m│\u001b[0m                                                                              \u001b[2m│\u001b[0m                  \n",
       "                 \u001b[2m╰──────────────────────────────────────────────────────────────────────────────╯\u001b[0m                  \n",
       "\n",
       "\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">[11/13/25 11:36:01] </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> Starting MCP server <span style=\"color: #008000; text-decoration-color: #008000\">'Elasticsearch GitHub Issues MCP'</span> with transport    <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">server.py:2050</span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         <span style=\"color: #008000; text-decoration-color: #008000\">'sse'</span> on <span style=\"color: #0000ff; text-decoration-color: #0000ff; text-decoration: underline\">http://0.0.0.0:8000/sse</span>                                        <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">              </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m[11/13/25 11:36:01]\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m Starting MCP server \u001b[32m'Elasticsearch GitHub Issues MCP'\u001b[0m with transport    \u001b[2mserver.py\u001b[0m\u001b[2m:\u001b[0m\u001b[2m2050\u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         \u001b[32m'sse'\u001b[0m on \u001b[4;94mhttp://0.0.0.0:8000/sse\u001b[0m                                        \u001b[2m              \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Started server process [47952]\n",
      "INFO:     Waiting for application startup.\n",
      "INFO:     Application startup complete.\n",
      "INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)\n",
      "INFO:pyngrok.process.ngrok:t=2025-11-13T11:37:09-0300 lvl=info msg=\"join connections\" obj=join id=2f547f1e02b9 l=127.0.0.1:8000 r=191.233.196.115:8612\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:     191.233.196.115:0 - \"POST /sse HTTP/1.1\" 405 Method Not Allowed\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pyngrok.process.ngrok:t=2025-11-13T11:37:10-0300 lvl=info msg=\"join connections\" obj=join id=f157e39aac9d l=127.0.0.1:8000 r=191.233.196.120:47762\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:     191.233.196.120:0 - \"GET /sse HTTP/1.1\" 200 OK\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pyngrok.process.ngrok:t=2025-11-13T11:37:10-0300 lvl=info msg=\"join connections\" obj=join id=5a9192136cfb l=127.0.0.1:8000 r=191.233.196.117:53796\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:     191.233.196.117:0 - \"POST /messages/?session_id=a8b8863d0264414f8cadb3694f26e121 HTTP/1.1\" 202 Accepted\n",
      "INFO:     191.233.196.117:0 - \"POST /messages/?session_id=a8b8863d0264414f8cadb3694f26e121 HTTP/1.1\" 202 Accepted\n",
      "INFO:     191.233.196.117:0 - \"POST /messages/?session_id=a8b8863d0264414f8cadb3694f26e121 HTTP/1.1\" 202 Accepted\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:mcp.server.lowlevel.server:Processing request of type ListToolsRequest\n",
      "INFO:pyngrok.process.ngrok:t=2025-11-13T11:47:43-0300 lvl=info msg=\"received stop request\" obj=app stopReq=\"{err:<nil> restart:false}\"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Server stopped\n"
     ]
    }
   ],
   "source": [
    "server = create_server()\n",
    "\n",
    "print(\"Starting MCP server...\")\n",
    "print(\"Server is running. To stop: Runtime > Interrupt execution\")\n",
    "print()\n",
    "\n",
    "\n",
    "def run_server():\n",
    "    server.run(transport=\"sse\", host=\"0.0.0.0\", port=8000)\n",
    "\n",
    "\n",
    "server_thread = threading.Thread(target=run_server, daemon=True)\n",
    "server_thread.start()\n",
    "\n",
    "print(\"Server started successfully!\")\n",
    "print(\"Your ngrok URL is ready to use in ChatGPT\")\n",
    "print(\"Keep this cell running...\")\n",
    "print()\n",
    "\n",
    "try:\n",
    "    while True:\n",
    "        time.sleep(1)\n",
    "except KeyboardInterrupt:\n",
    "    print(\"\\nServer stopped\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example: ChatGPT Interaction\n",
    "\n",
    "Here's an example of ChatGPT using the Elasticsearch connector to search through GitHub issues:\n",
    " - **Search tool:**\n",
    "\n",
    "    ![Search Example](./images/chatgpt-search-example.png)\n",
    " - **Fetch tool:**\n",
    "\n",
    "    ![Fetch Example](./images/chatgpt-fetch-example.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oNzxwv__hy8D"
   },
   "source": [
    "## Cleanup (Optional)\n",
    "\n",
    "Delete the Elasticsearch index to remove all demo data. \n",
    "**WARNING**: This permanently deletes all documents in the index. Only run this if you want to start fresh or clean up after the demo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7C6sin_gh2Be"
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    result = es_client.options(ignore_status=[400, 404]).indices.delete(\n",
    "        index=INDEX_NAME\n",
    "    )\n",
    "    if result.get(\"acknowledged\", False):\n",
    "        print(f\"Index '{INDEX_NAME}' deleted successfully\")\n",
    "    else:\n",
    "        print(f\"Error deleting index: {result}\")\n",
    "except Exception as e:\n",
    "    print(f\"Error: {e}\")"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.14.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
